1. Field of the Invention
The present invention relates to data transmission, and more particularly, to the testing of a JAVA™(Java) technology implementation to determine compliance with the Java specification.
2. Description of the Related Art
Currently, Java environments can be categorized into various Java technologies. A Java technology is defined as a Java specification and its reference implementation. Examples of Java technologies are Java 2 Standard Edition (J2SE), Java 2 Enterprise Edition (J2EE), Java 2 Micro Edition (J2ME), and Mobile Information Device Profile (MIDP). As with most other types of Java software, a new Java technology should be tested to assure consistency across multiple platforms. This testing is generally performed using compatibility testing.
Compatibility testing refers to the methods used to test an implementation of a Java technology specification in order to assure consistency across multiple hardware platforms, operating systems, and other implementations of the same Java technology specification. When this assurance is accomplished by means of a formal process, application developers can then be confident that an application will run in a consistent manner across all tested implementations of the same Java technology specification. This consistent specification-based behavior is a primary function of compatibility testing.
Compatibility testing differs from traditional product testing in a number of ways. Unlike product testing, compatibility testing is not primarily concerned with robustness, performance, or ease of use. The primary purpose of Java compatibility testing is to determine whether an implementation of a technology is compliant with the specification of that technology.
Compatibility test development for a given feature relies on a complete specification and reference implementation for that feature. Compatibility testing is a means of ensuring correctness, completeness, and consistency across all implementations of a technology specification that are developed. The primary goal of compatibility testing is to provide the assurance that an application will run in a consistent manner across all tested implementations of a technology.
To determine if the implementation of a particular Java technology is compliant with the specification for the particular Java technology, technology compatibility kits (TCK) may be used. A TCK is a suite of tests, tools, and documentation that allows an implementor of a Java technology specification to determine if the implementation is compliant with the specification.
A TCK typically includes a Test Harness, defined as the applications and tools that are used for test execution and test suite management, and a TCK Test Suite, which is the composite of the actual test cases in a TCK that are executed to test an implementation. A TCK can also include documentation that includes the specific TCK usage procedures, and the compatibility testing requirements that apply to the related technology release (usually in the form of a TCK user's guide). Also, a description of the TCK appeals process can be included, as well as an audit process, which is used to better ensure the integrity of a consistent self-testing compatibility program.
As mentioned above, a TCK usually includes a TCK test suite, which is a set of tests designed to verify that an implementation of a Java technology complies with the appropriate specification. Each test in a TCK test suite is composed of one or more test cases that are designated by a test description. A test case is the source code and accompanying information designed to exercise one aspect of a specified assertion. Accompanying information may include test documentation, auxiliary data files and other resources used by the source code. Some tests in the TCK test whether proper security checking is done before an operation is conducted according to the Java specification. Those tests insure that an operation can be successfully completed given enough permissions and some operations will fail by throwing security exceptions if not enough permissions are granted.
Unfortunately, the prior art handles security constraints in inconsistent ways in TCK tests. In a first prior art approach, a test lets the implementation throw security exceptions without catching them. This typically leads to test failures and generally forces the security constraints to be removed in order to pass the tests.
In second prior art approach, a test tries to install a security manager that grants the required permissions. If that fails, the test simply returns as if it is passed. Unfortunately, this approach does not take into account the possibility that the existing security manager may actually grant the required permissions, even if the desired security manager cannot be installed. Removal of a test-installed security manager may be easily forgotten thereby leading to a changed executing environment for other remaining tests. Also, different tests may install different security managers. If multiple tests are executed currently, one test may run with the security manager installed by another test. Code redundancy and duplication can arise if each test tries to install and remove a security manager and execute the security managers in a synchronized manner.
In a third prior art approach, a test catches the security exceptions as thrown in the first prior art example and checks whether the installed security manager actually grants the required permissions. If it does not, then the security exception is expected. Otherwise the security exception should not be thrown and the test returns as failed. This approach does not take into account that a security manager that grants the required permissions may be installed by the test as in the second prior art approach.
In view of the foregoing, there is a need for a more efficient and intelligent method of testing whether a Java technology implementation is compliant with the Java technology specification.